Skip to content

Incremental build optimizations#4082

Draft
atobiszei wants to merge 23 commits intomainfrom
atobisze_build_opt_5
Draft

Incremental build optimizations#4082
atobiszei wants to merge 23 commits intomainfrom
atobisze_build_opt_5

Conversation

@atobiszei
Copy link
Collaborator

Build Optimization: Reduce Dependencies, Compilation Unit Sizes, and Improve Build Graph Structure

Goal

Minimize transitive dependencies and preprocessed translation unit (TU) sizes across the OVMS codebase. Heavy headers like modelmanager.hpp, logging.hpp, status.hpp, and protocol-specific includes were being pulled into dozens of translation units that didn't need them. This inflated build times, reduced build parallelism, and made the dependency graph unnecessarily coupled.

Key results (measured on //src:ovms target, mp_on_py_on, using tools/compare_build_metrics.py --scopes all,non-test,test):

Primary KPI view(non-test files only):

Metric Before After Change
Dependency files measured 248 258 +10
Average total deps/file 57.5 49.8 -13.5%
Average internal deps/file 39.3 34.1 -13.3%
Files in >=201 deps bucket 34 25 -9
TU files measured 1,121 1,131 +10
Total preprocessed lines 60,572,349 59,798,140 -1.3%
Total preprocessed size (MB) 2202.7 2167.7 -1.6%
Average PP lines/file 54,034 52,872 -2.2%
Files in >=300001 PP-lines bucket 6 2 -4

Scope comparison (dependencies):

Scope Files Avg total deps/file Avg internal deps/file
All files 302 -> 312 69.8 -> 64.1 (-8.1%) 48.2 -> 44.7 (-7.2%)
Non-test files 248 -> 258 57.5 -> 49.8 (-13.5%) 39.3 -> 34.1 (-13.3%)
Test files 54 -> 54 125.9 -> 132.4 (+5.2%) 88.9 -> 95.5 (+7.3%)

Dependency buckets (non-test):

Bucket Before After Change
0-5 deps 37 38 +1
6-10 deps 34 35 +1
11-20 deps 63 66 +3
21-50 deps 54 59 +5
51-100 deps 26 35 +9
101-200 deps 0 0 0
>=201 deps 34 25 -9

PP-line buckets (non-test TU view):

Bucket Before After Change
0-10000 225 225 0
10001-25000 142 142 0
25001-50000 231 231 0
50001-100000 433 440 +7
100001-200000 61 73 +12
200001-300000 23 18 -5
>=300001 6 2 -4

Top individual TU reductions:

File Before After Reduction
model.cpp 296K 114K -61.5%
model_service.cpp 336K 153K -54.5%
modelmanager.cpp 337K 180K -46.6%
mediapipegraphdefinition.cpp 284K 160K -43.6%
dl_node.cpp 191K 107K -44.3%
dlnodesession.cpp 191K 107K -44.2%
mediapipefactory.cpp 234K 145K -38.0%
pipeline_factory.cpp 280K 175K -37.4%
pipelinedefinition.cpp 292K 190K -34.9%
kfs_grpc_inference_service.cpp 332K 220K -33.7%

Approach — High Level

1. Header Slimming via Forward Declarations and Include Discipline

Targeted the most included headers and removed unnecessary transitive dependencies:

  • status.hpp: Removed #include "logging.hpp" (50+ TUs no longer transitively pull in spdlog). Added minimal <fmt/format.h> for the fmt::formatter<StatusCode> specialization.
  • logging.hpp: Moved spdlog sink headers to logging.cpp (only needed in configure_logger()). Removed unused <fmt/ranges.h> (added only to 4 files that actually format containers). Removed logging.hpp from 12 headers that had zero logging symbol usage.
  • modelmanager.hpp: Removed <openvino/openvino.hpp> (replaced with forward-decl of ov::Core), removed <spdlog/spdlog.h>, <sys/stat.h>, model.hpp. Moved modelFactory() body to .cpp. This eliminates the entire OpenVINO umbrella header from 17+ TUs.
  • modelconfig.hpp: Removed <fstream>, <set>, anonymous_input_name.hpp. Moved isShapeAuto()/isShapeAnonymous() inline bodies to .cpp.
  • pipelinedefinition.hpp: Removed protobuf includes, narrowed OpenVINO include scope.
  • kfs_grpc_inference_service.hpp: Narrowed OpenVINO include.
  • mediapipegraphdefinition.hpp: Forward-declared servable types, removed iostream, moved parse_text_proto and status includes to .cpp.

2. Interface Extraction — Decoupling Subsystems from ModelManager

Introduced lightweight abstract interfaces so that subsystems depend on narrow contracts instead of the full ModelManager:

  • ServableNameChecker interface (servableExists()): Used by mediapipe, DAGs, and model loading code for name collision checking. Replaces direct ModelManager dependency in mediapipegraphdefinition.hpp/.cpp and mediapipefactory.hpp/.cpp.
  • MetricProvider interface (getMetricRegistry(), getMetricConfig()): Decouples metric access from ModelManager.
  • ServableType enum with bitmask operators: servableExists() now accepts a ServableType parameter (Model, Pipeline, Mediapipe) to specify which types to check, eliminating #if MEDIAPIPE_DISABLE guards and direct factory access.
  • ServableDefinition hierarchy:
    • ServableDefinition — pure virtual interface (getName, isAvailable)
    • SingleVersionServableDefinition — bridges ServableDefinition + Servable, provides unified waitForLoaded(), getStatus(), getInputsInfo(), getOutputsInfo(), getMetricReporter() etc.
    • Model, PipelineDefinition, MediapipeGraphDefinition all fit into this hierarchy
    • findServableDefinition() in ModelManager centralizes the model→pipeline→mediapipe lookup, replacing duplicated resolution cascades across 5+ frontend files.
    • Unified ServableDefinitionUnloadGuard replaces separate PipelineDefinitionUnloadGuard + MediapipeGraphDefinitionUnloadGuard.
  • StatusMetricReporter abstract base: Decouples metric reporting from model-specific reporter.

3. Mediapipe Target Decomposition

Split the mediapipe code from the monolithic ovms_lib target into focused Bazel targets:

  • //src/mediapipe_internal:libovms_mediapipe — Core mediapipe: factory, definition, executor, config. Dependencies restricted to specific //src: targets (no catch-all ovms_dependencies).
  • //src/mediapipe_internal:mediapipe_utils — String parsing utilities for mediapipe config.
  • //src/mediapipe_internal:node_initializer — Registry + interface for calculator initialization.
  • Frontend executor split: libovms_mediapipe_kfs_executor and libovms_mediapipe_http_executor as separate targets (KFS/HTTP graph executor impls). These depend on protocol-specific code; the core libovms_mediapipe does not.
  • Calculator deps remain at ovms_lib level (they are the integration point that links everything together).

4. NodeInitializer Registry Pattern

Replaced a ~250-line initializeNodes() function in mediapipegraphdefinition.cpp (a God function with #ifdef blocks and direct coupling to every subsystem) with a self-registering plugin pattern:

  • NodeInitializer interface: initialize(GraphSidePackets&), deinitialize(GraphSidePackets&), name().
  • NodeInitializerRegistry singleton: subsystems register themselves at static init time.
  • 7 self-registering initializers: llm_node_initializer.cpp, embeddings_node_initializer.cpp, rerank_node_initializer.cpp, python_node_initializer.cpp, image_gen_node_initializer.cpp, stt_node_initializer.cpp, tts_node_initializer.cpp.
  • Each initializer lives in its own subsystem directory (e.g., src/llm/, src/embeddings/) and self-registers via a global constructor. The core mediapipe code has zero knowledge of specific subsystems.
  • The old initializeNodes() body becomes a ~15-line loop: for (auto& init : registry) init->initialize(packets);.

5. Dead Include / Dead Code Cleanup

  • Removed 5 dead includes from core mediapipe (zero symbols used from KFS/TFS headers in mediapipegraphdefinition.cpp).
  • Removed dead PipelineFactory::pipelineDefinitionExists() (replaced by unified servableExists()).
  • Removed dead configAllowedLayouts static set from modelconfig.hpp.
  • Removed dead template createPipeline wrapper from ModelManager.
  • Moved PipelineFactory and MediapipeFactory to unique_ptr members in ModelManager (forward-declared in header).

Commit History

# Commit Description
1 2dd4e89 Decouple status.hpp from spdlog/fmt; slim down logging.hpp
2 8c5590c Remove logging.hpp from 12 headers with zero logging symbols
3 dd6571e Slim modelmanager.hpp — remove OpenVINO, spdlog, sys/stat, model.hpp
4 7bc73c2 Slim modelconfig.hpp — remove fstream, set, inline function bodies
5 08cd7e0 Introduce ServableNameChecker and MetricProvider interfaces
6 948229f Add ServableType flags for unified servableExists()
7 1aa6003 Move PipelineFactory to unique_ptr, forward-declare in header
8 b7d3f4e Remove pipelineDefinitionExists, replaced by servableExists
9 a09dd9f Introduce ServableDefinition hierarchy (Phase 1)
10 a9699ee Add findServableDefinition, refactor frontend resolution cascades
11 c9e6bfb Expand SingleVersionServableDefinition, unify guards + waitForLoaded
12 10bc0ef Remove unnecessary includes, move protobuf include to cpp
13 1e888cf Remove transitive includes — tensorinfo from servable.hpp, forward-declare servable types
14 7ff32a5 Remove protobuf includes from pipelinedefinition.hpp, narrow OpenVINO include
15 f0daff0 Split mediapipe targets, NodeInitializer registry, remove KFS deps from core
16 580ebd Move mediapipe BUILD targets to src/mediapipe_internal/BUILD, fix copyright + includes

Files Changed

~100 files across 16 commits. The changes are strictly additive in terms of functionality (zero behavioral changes) and subtractive in terms of coupling.

New Files

File Purpose
src/servable_name_checker.hpp ServableNameChecker interface
src/metric_provider.hpp MetricProvider interface
src/servable_type.hpp ServableType enum with bitmask operators
src/servable_definition.hpp ServableDefinition pure virtual base
src/single_version_servable_definition.hpp/.cpp Unified servable definition bridging class
src/servable_definition_unload_guard.hpp/.cpp Unified unload guard
src/tensorinfo_fwd.hpp Forward declarations for TensorInfo types
src/mediapipe_internal/BUILD Mediapipe-specific Bazel targets
src/mediapipe_internal/node_initializer.hpp/.cpp NodeInitializer interface + registry
src/mediapipe_internal/graph_side_packets.hpp Extracted side packets struct
src/llm/llm_node_initializer.cpp LLM self-registering initializer
src/embeddings/embeddings_node_initializer.cpp Embeddings self-registering initializer
src/rerank/rerank_node_initializer.cpp Rerank self-registering initializer
src/python/python_node_initializer.cpp Python self-registering initializer
src/image_gen/image_gen_node_initializer.cpp Image gen self-registering initializer
src/audio/speech_to_text/stt_node_initializer.cpp STT self-registering initializer
src/audio/text_to_speech/tts_node_initializer.cpp TTS self-registering initializer

- Remove unused #include "logging.hpp" from status.hpp (50+ files no longer
  transitively pull in spdlog). Add minimal #include <fmt/format.h> for
  fmt::formatter<StatusCode> specialization, with @fmtlib dep in BUILD.
- Remove #include <fmt/ranges.h> from logging.hpp; add it only to the 4
  files that actually format containers (imagegen_init.cpp,
  openai_completions.cpp, assisted_decoding_test.cpp, llmnode_test.cpp).
- Move <spdlog/sinks/basic_file_sink.h> and <spdlog/sinks/stdout_sinks.h>
  from logging.hpp to logging.cpp (sinks only needed in configure_logger()).
- Add explicit #include "logging.hpp" to color_format_configuration.cpp and
  precision_configuration.cpp that relied on the transitive include.

Verified: //src:ovms and //src:ovms_test build with mp_on_py_on,
mp_on_py_off, and mp_off_py_off configs.
Removed #include logging.hpp from 12 headers that had zero logging
symbol usage. Added explicit #include <fmt/format.h> to 3 headers that
have fmt::formatter specializations (grpc_utils.hpp, modelversionstatus.hpp,
nodeinfo.hpp). Added explicit #include logging.hpp to 2 files that lost
transitive include (tensor_conversion.hpp, tensor_conversion_common.cpp).
Updated BUILD deps: replaced libovmslogging with @fmtlib where only
fmt was needed.
…odel.hpp

- Removed #include <openvino/openvino.hpp>, replaced with forward
  declaration of ov::Core (only used as unique_ptr member)
- Removed #include <spdlog/spdlog.h> (unused in header)
- Removed #include <sys/stat.h> (unused in header)
- Replaced #include model.hpp with modelconfig.hpp + forward decl of
  Model. Moved modelFactory() body to modelmanager.cpp.
- Added #include <openvino/openvino.hpp> to modelmanager.cpp
- Added #include model.hpp to 4 .cpp files and 4 test files that
  used Model via shared_ptr through the transitive include.

This eliminates the entire OpenVINO umbrella header and spdlog from
the 17 translation units that include modelmanager.hpp.
…name

- Removed #include <fstream> (unused in header, added to .cpp)
- Removed #include <set> and dead static member configAllowedLayouts
  (declared but never defined or used anywhere)
- Removed #include anonymous_input_name.hpp, moved isShapeAuto(),
  isShapeAnonymous(), isShapeAnonymousFixed(), anyShapeSetToAuto()
  bodies from inline in header to modelconfig.cpp
- Added #include <fstream> and anonymous_input_name.hpp to .cpp
…to decouple mediapipe from ModelManager

- Create ServableNameChecker interface (servableExists) and MetricProvider
  interface (getMetricRegistry, getMetricConfig)
- ModelManager inherits both interfaces
- MediapipeFactory uses unique_ptr, forward-declared in modelmanager.hpp
- mediapipegraphdefinition.hpp/cpp depend on ServableNameChecker, not ModelManager
- mediapipefactory.hpp/cpp depend on interfaces, not ModelManager
- Remove dead parameters: retire(), create(), retireOtherThan(), revalidatePipelines()
- Add explicit mediapipefactory.hpp includes where needed (4 src + 3 test files)
…hecking

- Add ServableType enum (Model, Pipeline, Mediapipe) with bitmask operators
- servableExists() accepts ServableType check param to specify which types to check
- Mediapipe validate: checks Model | Pipeline (skips self)
- DAG validate: checks Model | Mediapipe (skips self), removes mediapipefactory.hpp include
- Model loading: checks Pipeline | Mediapipe (skips self), removes #if MEDIAPIPE_DISABLE guards
- All three servable types now use the same unified servableExists() function
…apper

- Change PipelineFactory member from value to unique_ptr in ModelManager
- Forward-declare PipelineFactory, Pipeline, ModelInstance, ModelInstanceUnloadGuard in modelmanager.hpp
- Move pipeline_factory.hpp include from modelmanager.hpp to modelmanager.cpp
- Remove template createPipeline wrapper from ModelManager
- Callers now use getPipelineFactory().create() directly
- Add explicit pipeline_factory.hpp includes to files that need it
- Add ServableDefinition pure virtual interface (getName, isAvailable)
- Add SingleVersionServableDefinition bridging ServableDefinition + Servable
- Model implements ServableDefinition directly
- PipelineDefinition inherits SingleVersionServableDefinition (removes pipelineName member)
- MediapipeGraphDefinition inherits SingleVersionServableDefinition (removes name member)
- Add Bazel targets for new headers
…cades

- Add ModelManager::findServableDefinition() centralizing model->pipeline->mediapipe lookup
- Refactor KFS getModelReady to use findServableDefinition (removes nested #if blocks)
- Refactor KFS ModelMetadataImpl to use findServableDefinition
- Refactor model_service getModelStatus to use findServableDefinition
- Refactor CAPI OVMS_GetServableState to use findServableDefinition
- Remove unused duplicate BUILD targets: libovmshttpservermodule, libovms_dags_pipelinedefinition
- Fix copyright year in single_version_servable_definition.hpp
- Expand SingleVersionServableDefinition with virtual methods:
  getStatus(), getInputsInfo(), getOutputsInfo(), getMetricReporter(),
  notLoadedYetCode(), notLoadedAnymoreCode()
- Unified waitForLoaded() in SingleVersionServableDefinition base class
- Add StatusMetricReporter abstract base to model_metric_reporter.hpp
- Unify ServableDefinitionUnloadGuard (replaces PipelineDefinitionUnloadGuard
  + MediapipeGraphDefinitionUnloadGuard)
- Remove mediapipe includes from model_service.cpp and capi.cpp
- Unify KFS buildResponse for pipeline/mediapipe paths
- Add getServableDefinitionNames() to ModelManager
- Add tensorinfo_fwd.hpp for forward declarations
- Fix dangling reference: use getName() instead of constructor parameter
  in PipelineDefinition and MediapipeGraphDefinition constructors
…forward-declare servable types in mediapipegraphdefinition.hpp, remove iostream
…w OpenVINO include in kfs_grpc_inference_service.hpp, move mediapipe parse_text_proto/status to cpp
…e KFS deps from core

Phase 3E: Split libovms_mediapipe from ovms_lib into separate target
Phase 3F: Extract NodeInitializer registry pattern from initializeNodes()
  - Created node_initializer.hpp/.cpp with NodeInitializer interface + registry
  - Created graph_side_packets.hpp extracted from mediapipegraphdefinition.hpp
  - 7 self-registering concrete initializers in their calculator directories
  - Refactored ~250 line God function to ~15 line registry loop
Phase 3G: Split frontend executors from core mediapipe
  - libovms_mediapipe (core: factory, definition, executor, config)
  - libovms_mediapipe_kfs_executor (kfs_graph_executor_impl)
  - libovms_mediapipe_http_executor (http_graph_executor_impl)
Phase 3H: Remove dead KFS/TFS deps from core mediapipe
  - Removed 5 dead includes (zero symbols used)
  - Replaced //:ovms_dependencies with specific rapidjson + mediapipe deps
  - Calculator deps stay at ovms_lib level (not in libovms_mediapipe)
  - Node initializers use logging.hpp instead of direct spdlog
  - Fixed missing rapidjson/document.h in rerank_servable.hpp
@atobiszei atobiszei changed the title Atobisze build opt 5 Incremental build optimizations Mar 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant